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ABSTRACT 


Improvement  in  the  process  of  acquiring  major  weapon  systems  has 
been  the  subject  of  analyses  and  policy  recommendations  for  several 
decades.  While  system  costs  have  increased  as  weapon  systems  have 
grown  more  complex,  for  programs  of  comparable  duration  and  technical 
difficulty  the  extent  of  cost  growth  over  original  estimates  has  not 
significantly  improved.  Furthermore,  there  is  little  evidence  that 
procedural  changes  in  recent  practices  of  system  acquisition  will  pro¬ 
duce  important  improvements. 

To  obtain  a  better  understanding  of  this  evidence,  a  model  captur¬ 
ing  the  dimensions  of  and  controls  over  an  acquisition  program  is 
developed  and  analyzed.  The  model  includes  various  dimensions  of  program 
performance--not  only  system  performance  and  program  schedule  but  also 
development  and  system  cost--as  well  as  acquisition  strategies--com- 
prised  of  combinations  of  study,  test,  and  demonstration  activities; 
contractor  environments  and  contract  types;  and  sets  of  decision  points 
for  program,  technological,  or  threat  reassessment.  Choice  among 
strategies  can  be  determined  within  a  budget  constraint  by  the  utility 
of  performance  dimensions  actually  achieved  to  counter  threats  that 
actually  materialize.  The  empirical  results  concentrate  on  determina¬ 
tion  of  the  technological  advancement  of  a  system  and  on  investigation 
of  the  extent  to  which  program  performance  flexibility  is  constrained 
by  the  acquisition  strategy  selected. 
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CHOICE  AMONG  STRATEGIES  FOR  SYSTEM  ACQUISITION 

* 

Alvin  J.  Harman 
The  Rand  Corporation 

Improvement  in  the  process  of  acquiring  major  weapon  systems  has 
been  the  subject  of  analyses  and  policy  recommendations  for  several 
decades  [see,  for  example,  Klein  (1962),  Peck  and  Scherer  (1962), 
Marschak,  et  al .  (1967),  Perry,  et  al .  (1971)].  While  system  costs 
have  increased  as  weapon  systems  have  grown  more  complex,  for  programs 
of  comparable  duration  and  technical  difficulty,  the  extent  of  cost 
growth  over  original  estimates  has  not  significantly  improved  [Harman 
(1970)]. 

The  objective  of  this  paper  is  to  suggest  a  new  empirical  formula¬ 
tion  by  which  the  influence  on  program  performance  of  strategy  variations 
can  be  measured.  Major  performance  attributes  of  a  program  include 
the  technical  performance  of  the  end  product  (e.g.,  its  ability  to 
perform  one  or  several  types  of  missions),  the  schedule  within  which 
it  is  delivered  to  operational  status,  and  the  price  within  which  the 
above  characteristics  are  achieved.  As  a  matter  of  definition,  a 
strategy  is  the  mix  of  various  elements  of  program  structure  to  achieve 
the  performance  outcome  (e.g.,  to  achieve  maximum  technical  advance¬ 
ment  for  a  given  price  and  schedule,  or  minimal  cost  for  a  given  tech¬ 
nical  and  schedule  performance) .  The  elements  may  include  studies 
during  the  conceptualization  of  the  new  system,  various  test  or  demon¬ 
stration  activities  on  either  prototype  or  early  production  hardware, 
sets  of  decision  points  for  technological  or  threat  reassessment  and 

program  restructuring,  alternative  contractor  environments  and  contract 

- 

Any  views  expressed  in  this  paper  are  my  own.  They  should  not 
be  interpreted  as  reflecting  the  views  of  The  Rand  Corporation  or  the 
official  opinion  or  policy  of  any  of  its  governmental  or  private  research 
sponsors.  Papers  are  reproduced  by  The  Rand  Corporation  as  a  courtesy 
to  members  of  its  staff. 

This  paper  was  presented  at  the  Winter  Meetings  of  the  Econometric 
Society  in  New  Orleans,  December  27-29,  1971.  I  would  like  to  thank 
A.  J.  Alexander,  M.  Berman,  H.  H.  Harman,  J.  Koehler,  J.  R.  Nelson, 

R.  R.  Nelson,  R.  Perry,  R.  Shishko,  and  G.  K.  Smith  for  their  comments 
and  suggestions  at  various  stages  of  research;  and  S.  Henrichsen  and 
E.  0.  Watkins  for  their  assistance  in  data  compilation  and  statistical 
analysis.  Needless  to  say,  any  errors  are  solely  my  responsibility. 
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types,  and  methods  for  verification  of  contract  completion. 

While  exceedingly  complex  in  practice,  conceptually  it  is 
straightforward  that  choice  among  strategies  (involving  different  com¬ 
binations  of  strategy  elements)  can  be  determined  within  a  budget  con¬ 
straint  by  the  utility  of  technical  ano  /.chedule  performance  dimensions 
actually  achieved  to  counter  threats  rrat  actually  materialize.  Rather 
than  attempting  to  deal  explicitly  w^.i  optimality  conditions  [see 
Shishko  (1971)],  empirical  evidence  j*’  strategy  differences  can  be 
investigated.  One  approach  would  be  toe  study  of  radically  different 
strategies  in  use  for  programs  exer  t  from  the  usual  DoD  practices, 
or  in  use  for  similar  programs  abroa ?  [Alexander  (1970),  Perry,  et  al. 
(1971)].  This  approach  has  the  advantage  of  potentially  revealing 
not  only  strategies  with  more  predictable  outcomes,  but  also  with 
lower  cost  outcomes.  On  the  othr^  hand,  it  is  difficult  to  imagine 
how  such  radically  different  strategies  can  be  implemented  within  the 
current  OoO  institutions.  I  will  therefore  focus  on  methodological 
considerations  and  empirical  evidence  (from  DoD  programs  that  employed 
standard  practices)  for  selection  criteria  among  standard  strategy 
elements. 


. . ~ if 

A  number  of  these  elements  have  been  the  subject  of  previous 
theoretical  and  empirical  investigations  [see  Nelson  (1961),  Klein, 
et  al.  (1971),  Scherer  (1964,  1966),  Cross  (1968),  Fisher  (1968), 
Hall  (1969)]. 
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I .  AN  OVERVIEW  OF  THE  WEAPON  ACQUISITION  PROCESS 

As  I  have  described  in  greater  detail  elsewhere,  the  overall  at¬ 
tributes  of  program  performance  can  be  analyzed  rather  easily  provided 
that  certain  assumptions  are  correct  [Harman  (1970)].  The  experience 
in  acquiring  major  weapon  systems  can  be  sunmarized  if  the  main  strat¬ 
egy  attributes  have  not  changed  radically  within  the  periods  of  in¬ 
terest.  Furthermore,  in  the  process  of  acquiring  new  military  systems, 
it  is  reasonably  realistic  to  assume  that  the  system  of  equations  de¬ 
scribing  the  technical,  schedule,  and  price  outcomes  is  block  recursive; 
that  thi  actual  price  implications  of  the  processes  determining  tech¬ 
nical  and  schedule  performance  are  not  reflected  either  directly  (as 
a  variable  in  the  equations)  or  implicitly  (through  correlated  distur¬ 
bances).  The  case  for  this  assumption  rests  largely  on  the  subset  of 
military  R&D  under  mvestigation--rather  far  along  in  the  R&D  process 
to  the  point  of  "system  development."  While  the  choice  among  early 
R&D  projects  is  likely  to  reflect  price  versus  technical  performance 
considerations,  the  choices  among  reasonably  well-defined  systems  tend 
to  be  dominated  by  technical  and  schedule  considerations.  Furthermore, 
even  if  attempts  have  bean  made  during  the  planning  stage  to  make  cost- 
technical  tradeoffs,  the  changes  authorized  and  inflexibility  main¬ 
tained  during  the  execution  of  past  programs  have  tended  to  vitiate 
these  attempts.  On  the  other  hand,  costs  ultimately  borne  by  the  pro¬ 
curing  service  are  not  completely  responsive  to  severe  technical  and 
schedule  requirements;  other  levels  and  branches  of  government  that 

have  not  attempted  to  control  the  other  performance  features  do  exer- 

★ 

cise  limitations  of  greater  or  lesser  severity  on  costs.  As  a  broad 
overview  of  the  process,  therefore,  I  have  investigated  cost  consequences 
of  other  aspects  of  program  performance,  and  have  t-i'ioyed  ordinary 
least  squares. 

Tnroughout  this  paper,  I  am  using  "costs"  to  denote  the  total 
program  expenditure  by  the  procuring  service,  and  not  the  total  cost 
to  the  developer.  The  latter  will  be  simultaneously  determined  with 
technical  and  schedule  performance,  while  the  former  may  be  signifi¬ 
cantly  influenced  by  that  branch  of  the  service  as  well  as  other 
elements  of  government,  resulting  in  either  a  profit  or  loss  to  the 
developer. 
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The  model  itself  deals  with  the  uncertainty  of  cost  outcomes;  the 
measure  of  primary  interest  has  been  the  cost  factor  (F),  the  ratio  of 
actual  costs  {development  and  production)  of  the  completed  program  to 
the  estimate  of  these  costs  made  at  the  time  of  program  approval.  The 
hypotheses— that  underestimate  or  growth  of  actual  costs  would  be 
increasingly  likely  for  more  complex  and/or  technologically  advanced  sys¬ 
tems,  and  that  the  dispersion  would  tend  to  increase  with  these  biases— 
were  tested  using  the  equation 

log  F  =  a  +  b  L  •  E  (1) 

in  which  L  is  the  development  program  length  in  months  and  E  represents 

the  intensity  of  effurt  per  month  of  development  necessary  to  accomplish 

the  program.  This  intensity  should  reflect  the  performance  advancements 

of  the  system  to  be  achieved  within  the  schedule  provided.  The  model 

was  used  to  investigate  aircraft  and  missile  acquisitions  in  the  1950s 

and  1960s,  and  to  see  if  any  significant  difference  in  the  system  acqui- 

* 

sition  process  in  these  decades  could  be  detected. 

Since  no  direct  measurement  of  E  was  available,  two  functions  of 

ifk 

a  subjective  measure  of  technological  advance  sought  (A)  for  the  sys¬ 
tem  (as  a  surrogate  for  the  performance  advancements)  and  L  were  con¬ 
sidered,  The  function  selected  was  simply 

E  =  f (A)  (2) 


That  is,  E  was  represented  by  a  mono tonic  function  of  the  technological 
advancement  subjective  measure.  Moreover,  a  stretching  of  this  A  di- 

ictfk 

mension  provided  the  best  fit.  A  few  of  the  empirical  results  are 
reproduced  in  Table  1  for  convenient  reference. 


The  measure  of  l  for  the  1950s  systems  was  actually  the  time 
from  earliest  estimate  (on  which  the  cost  factor  was  based)  to  develop¬ 
ment  completion  (initial  operational  delivery,  10D).  A  better  measure 
was  found  for  later  analysis,  see  below. 


For  a  description  of  the  surveys  that  produced  this  measure 
see  [Harman  (1970)  Section  III]. 


The  A  variable  ranged  from  about  5  to  16 
of  0  ("off  the  shelf")  to  20  ("basically  new  and 
system  design").  Four  monotonic  transforms  of  A 
A,  As  e  . 


on  an  arbitrary  scale 
radically  different 
were  tested:  log  A, 
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Table  1 

SUMMARY  RESULTS  OF  COST  FACTOR  MODEL3 


Sample 

a 

b 

R2 

Samp I e 
Size 

structure  linear  in  "A" 
loqF=a+bL*A 

1950s 

.021 

(.1) 

.00078 

(3.4) 

.39 

r 

21 

1960s 

.008 

(.8) 

.00035 

0.5) 

.19 

12 

1 

Combined 

-.007 

(-.1) 

.00076 
i  (4.5) 

.39 

1  33 

Structure  with  SI 
log  F  = 

Wretched 
a  +  b  L 

"A”  Dimension 
•eA 

1950s 

.268 

(2.2) 

.15E-08 

(4.2) 

.48 

21 

1960s 

.180 

(4.5) 

.33E-08 

(4.0) 

.62 

12 

Combi ned 

.240 

(3.2) 

.15E-08 

ilzlj 

.51 

33 

at-statistics  are  presented  in  paren¬ 
theses  below  coefficient  estimates. 
Source:  Harman  (1970)  Table  6. 


The  conclusions  of  the  study  were  summarized  as  follows: 

The  coefficient  estimates  in  the  various  structures 
for  the  model  can  be  described  as  characterizations  of  the 
system  acquisition  process  in  each  of  the  two  decades. 

Average  values  of  the  development  characteristics— [L]  ... 
for  development  program  length  and  "A"  being  a  measure 
of  technological  advance  sought—characterize  the  way  in 
which  developments  have  differed  between  the  decades. 

The  statistical  tests  discussed  above  led  to  the  implica¬ 
tion  that  there  has  been  no  significant  difference  in 
the  process  between  the  two  decades;  one  even  finds  that 
the  process  seems  to  have  'deteriorated'  for  some  types 
of  development  programs  in  the  sense  that  for  a  given  set 
of  development  program  character  is tics  the  cost  factor  for 
the  1960s  imolied  by  the  model  would  be  higher  than  for  the 
1950s.  Despite  this  implication  for  the  process,  the  pro¬ 
grams'  outcomes— wb'ch  can  be  characterized  as  having  re¬ 
sulted  from  programs  in  the  1960s  that  were  somewhat  shorter 
and  of  somewhat  lower  technological  difficulty  on  the  average- 
have  shown  a  'typical*  1960s  program  to  have  a  somewhat 
lower  cost  factor  than  a  'typical'  1950s  program.  [Hannan 
(1970)  pp,  42-43,  notation  slightly  modified,  original  italics]. 
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There  are  two  features  of  these  results  that  I  will  pursue  below. 

First,  the  "technological  advance  sought"  for  a  system  plays  a  crucial 

role  in  explaining  cost  uncertainty,  and  it  is  particularly  the  systems 

with  high  values  for  A  that  have  experienced  large  and  essentially 

■* 

unpredictable  cost  growth.  However,  the  measure  used  was  obtained 
subjectively  ex  post.  Before  a  variable  like  A  can  be  used  as  a  sig¬ 
nal  for  strategy  selection— namely,  for  a  high  "A"  program,  find  a 
different  and  hopefully  better  strategy  than  has  been  used  in  recent 
experience— a  more  objective  and  potentially  ex  ante  measure  of  "tech¬ 
nological  advance  sought"  is  highly  desirriola. 

The  second  feature  to  be  considered  below  relates  to  the  fact 
that  the  cost  factor  model  assumes  that  the  strategy  for  system  acqui¬ 
sitions  has  been  essentially  the  same  for  all  programs  considered. 

While  this  has  been  borne  out  to  some  extent- by  the  results,  it  should 
still  be  revealing  to  refine  the  approach  to  one  in  which  the  impact 
of  variations  in  different  elements  of  the  strategy  could  be  considered. 

- * - 

In  the  results  of  the  1970  study,  this  conclusion— a  consequence 
of  the  selection  of  e^  as  the  most  appropriate  stretching  of  the  A 
dimension— had  to  be  based  largely  on  the  1950s  sample  in  which  14  of 
the  21  items  in  the  sample  had  A's  larger  than  12  (the  1960s  sample 
had  only  one  of  12).  Since  that  time,  the  1960s  sample  has  been  ex¬ 
panded  and  updated.  The  sample  now  includes  5  of  19  systems  with  A's 
larger  than  12  and  all  results  are  essentially  the  same. 
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I I .  TOWARD  ALTERNATIVE  STRATEGIES* 

Quantitative  Assessment  of  Technological  Advance 

In  an  effort  to  see  if  a  measure  comparable  to  the  "A"  could  be 
obtained,  I  have  investigated  technological  advancement  of  fighter  air¬ 
craft  in  greater  detail.  The  objective,  in  effect,  is  to  explore  the 
schedule-technical  performance  block  of  the  recursive  system  of  equa¬ 
tions  describing  program  outcomes.  While  a  detailed  investigation  of 
each  of  these  aspects  of  the  acquisition  process  would  be  well  worth 
undertaking,  for  the  present  I  have  confined  myself  to  obtaining  a 
summarization  of  the  observed  outcomes  of  this  block  of  the  model. 

The  basic  premise  for  the  formulation  to  follow  is  that  a  "most 
advanced"  system  can  be  identified  from  among  all  those  developed  prior 
to  the  beginning  of  each  program  to  be  investigated.  Such  a  system  can 
then  be  used  as  benchmark  for  assessing  the  technological  advance  of 
the  subsequent  development  program.  The  method  of  selecting  this  "most 
advanced"  system  will  be  based  on  the  trend-of- technology  methodology 
employed  by  Alexander  and  Nelson  in  their  analysis  of  technological  ad¬ 
vance  in  jet  engines.  The  entire  procedure  is  sketched  in  Fig.  1.  The 
trend  methodology  involves  predicting  the  time  at  which  a  development 
program  will  come  to  fruition  based  on  the  performance  of  the  completed 
system.  In  effect,  the  assumption  is  made  that  the  shape  of  the  trade¬ 
off  surface  among  the  performance  parameters  has  remained  stable  over 
the  period  but  has  shifted  to  higher  and  higher  levels  of  achievable 
technology  through  time.  The  plotted  points  in  Fig.  1  represent  the 
actual  development  endings  relative  to  those  predicted  (calculated) 
from  the  regression  equation,  based  on  the  performance  the  system  has 
achieved.  A  point  on  the  45°  line  would  represent  a  system  completing 
development  just  when  it  would  be  expected  to,  based  on  the  trend-of- 
technology  equation.  Similarly,  points  above  the  line  can  be  inter¬ 
preted  as  representing  systems  with  technology  ahead  of  their  time  and 

— 

The  empirical  results  in  this  section  are  quite  preliminary  due 
to  questions  of  accuracy  and  quality  of  the  data  that  are  still  being 
resolved.  I  believe,  however,  that  the  results  do  yield  some  insights 
into  the  usefulness  of  the  methodologies. 

The  selection  of  a  benchmark  from  such  a  technological  tradeoff 
surface  is  assumed  to  yield  a  system  "closest"  to  the  set  of  perfor¬ 
mance  characteristics  of  the  next  system  to  enter  development.  This 
assumption  is  necessary  in  making  the  rates  of  advancement  calculations 
(see  p.  10). 


Calculated  end  of  development 
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Fig.  1  --  Determining  the  Benchmark  "Most  Advanced"  System 
from  a  Trenci-of-Technology  Equation 


conversely  for  points  below  the  45°  line.  For  the  current  analysis,  at 
any  development  program  beginning  ,  the  most  recent  program  (in  the 
case  of  a  tie)  with  the  highest  "calculated"  end  of  development  (based 
on  the  equation  using  attained  performance  parameters)  will  be  taken  to 

ft 

be  the  "most  advanced"  prior  system. 

The  trend-of- technology  equation  used  for  the  remainder  of  the 

** 

analysis  of  fighter  aircraft  is 

IOD  =  175  +  lllZns  +  2.4Znw  -  88Znc 
(.2)  (2.5)  (.09)  (-1.0) 


where 

IOD  =  months  since  the  beginning  of  1940  to  the  initial  opera¬ 
tional  delivery 

s  =  maximum  speed 

w  =  empty  weight 

c  «  service  ceiling 

wl  c  wing  loading  (gross  weigK/wing  area) 

de  =  delivery  efficiency  (gross  weight/empty  weight) 

cr  =  climb  rate 

The  coefficients  that  are  even  marginally  significant  are  plausible 
in  interpretation— "most  advanced"  (latest)  aircraft  would  have  higher 
speed,  lower  wing  loading  (and  thus  higher  maneuverability),  greater 
delivery  efficiency,  and  a  greater  climb  rate.  Only  the  ceilinq  co¬ 
efficient  seems  to  be  incorrect  in  sign. 

Given  this  equation  and  the  definition  of  "state-of-art"  in  Fig. 
1,  the  two  main  Features  necessary  to  assess  the  technological  advance 


It  would  also  be  interesting  to  experiment  with  allowing  the 
benchmark  of  prior  system  to  overlap  somewhat  in  development  program 
with  a  new  program  conducted  by  the  same  developer.  On  the  other 
hand,  it  might  be  useful  to  insist  on  a  minimum  length  of  time  between 
systems  before  such  a  system  could  be  considered  prior  to  a  current 
development  (i.e.,  t.  prior  to  the  actual  beginning  of  development). 

t-statistics  are  presented  in  parentheses  below  coefficient 
estimates. 


-  88Znwl  +  150Znde  +  50Zncr  /0\ 
(-1,3)  (1.0)  (2.4) 

R2  =  .81 
Sample  size  =  18 
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for  a  new  development  are  the  performance  dimensions  sought  and  the 
length  of  the  development  program— that  is,  the  length  of  time  avail¬ 
able  to  achieve  those  new  performance  dimensions.  Of  course,  it  is 
possible  that  some  of  the  "technology"  embodied  in  this  new  system 
will  not  have  been  developed  specifically  for  it.  With  a  longer  gap 
between  the  end  of  one  development  program  and  the  beginning  of  another, 
the  possibility  increases  that  scientific  breakthroughs  or  less  dra¬ 
matic  shifts  in  technology  from  other  systems  or  programs  may  be  ap¬ 
plicable  to  this  new  program.  I  deal  with  this  other  means  of  "ad¬ 
vancing  technology"  shortly. 

The  technological  advance  required  in  a  particular  program,  then, 
is  a  function  of  the  growth  of  each  of  the  performance  dimensions  of 
the  new  system  from  the  performance  dimensions  already  available  in 
the  "most  advanced"  system  in  the  force.  If  P^l)  is  the  ith  perfor¬ 
mance  attribute  characteristic  of  the  "most  advanced"  recent  develop¬ 
ment,  the  P ( 2 )  is  the  corresponding  performance  dimension  for  the  new 
development  program,  then  the  rate  of  growth  in  that  performance  di¬ 
mension  in  the  course  of  development  is  given  by 


flog  P.(2)  -  log  P,(l)' 

t;  mn - ; 


(4) 


That  is,  r.j  is  the  annual  compound  rate  of  growth  of  the  ith  perfor¬ 
mance  dimension  necessary  to  achieve  P^(*2)  during  the  development  pro¬ 
gram  of  length,  (L/12)  years. 


The  choice  of  compound  growth  is  based  on  the  premise  that  some 
of  the  early  result',  of  a  development  program  are  useful  in  the  formu¬ 
lation  of  later  development  efforts.  The  extent  of  compounding  is 
somewhat  arbitrary— running  the  gamut  from  simple  growth  rates  (*.e., 
no  compounding,  which  can  be  thought  of  as  all  components  of  the  proj¬ 
ect  developed  in  parallel  and  assembled  at  the  very  end,  so  no  solution 
early  in  the  development  is  helpful  in  a  later  problem)  through  an¬ 
nual  compound  growth  rates,  monthly  compound  growth  rates,  to  Instan¬ 
taneous  compounding  (i.e.,  every  "idea"  or  thought  by  anyone  in  the 
development  effort  may  be  helpful  at  a  later  point  in  the  development). 
The  annual  compounding  is  an  intermediate  choice  between  these  extremes. 
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Even  after  using  all  of  the  major  performance  dimensions  for  a 
particular  type  of  weapon  system,  there  are  several  aspects  of  tech¬ 
nological  advance  that  are  certainly  not  captured  by  these  rates  of 
growth.  As  mentioned  earlier,  one  important  aspect  is  the  possible 
availability  of  research  or  scientific  advancement  not  associated 
with  the  development  of  a  particular  weapon  system.  The  length  of 
time  between  the  "-vast  advanced"  development  program  completion 

and  the  beginnir  i  of  the  current  one  can  be  identified  by  the  slack 
★ 

variable,  SL.  One  might  also  want  to  capture  major  technological  bar¬ 
riers  that  are  crossed  in  the  course  of  particular  development  pro¬ 
grams.  For  example,  to  differentiate  aircraft  designed  to  operate  at 
supersonic  speeds,  the  variable  rate  of  growth  of  maximum  speed  could 
be  dichotomized  into  a  subsonic  category  and  a  supersonic  category. 

For  the  current  analysis  only  the  r^'s  and  the  SL  variable  are  included 
as  the  set  of  variables  describing  technological  advance  of  a  program. 

It  would  be  quite  plausible  to  use  this  entire  set  of  variables 
as  the  characterization  for  the  observed  technological  advance  outcomes 
of  the  technical  and  schedule  performance  block  of  the  model.  However, 
it  seems  equally  reasonable  to  postulate  underlying  characteristics 
of  the  development  process  that  produce  this  set  of  outcomes.  For  ex¬ 
ample,  progress  in  materials,  aerodynamics,  and  engines  largely  deter¬ 
mines  tne  progress  in  most  observable  measures  of  technical  performance. 
To  take  this  last  step,  of  parsimonious  description  of  such  underlying 
dimensions  of  the  technological  advance,  I  have  employed  factor  analysis 

* 

It  might  be  valuable  to  distinguish  between  this  slack  variable 
between  development  of  different  fighters  and  a  second  slack  variable 
identifying  the  length  of  time  between  development  programs  by  a  par¬ 
ticular  developer.  This  might  capture  some  of  the  momentum  aspect  of 
maintaining  a  design  team  that  has  recently  learned  from  past  design 
effort  and  could  apply  its  recent  experience  to  design  of  subsequent 

systems. 

** 

The  MINRES  method  of  initial  solution  (a  least  squares  solution 
for  the  factor  loadings  and  communal i ties  simultaneously)  and  DIRECT 
OBLIMIN  (an  oblique)  rotation  have  been  used.  For  a  description  of  the 
similarity  between  simultaneous  equation  estimation  and  factor  analysis, 
see  [A.  S.  Goldberger  (1971)]. 

Of  course,  the  degree  of  success  in  achieving  such  a  description 
of  underlying  dimensions  depends  on  the  corrections  of  the  benchmark 
selection  as  well  as  the  appropriateness  of  the  linear  factor  analysis 
model . 


** 
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The  rotated  factor  loadings  are  presented  in  Table  2. 

Table  2 

FACTOR  ANALYSIS  RESULTS 


Variables 

Hi 

um 

rs 

.718 

rw 

-.591 

-.389 

.523 

rc 

.079 

.912 

.844 

rwl 

.955 

.978 

rde 

.386' 

-.497 

.378 

rcr 

.669 

.567 

SL 

.595 

-.056 

.354 

Variance  captured 
by  factors 

2.76 

1.68 

Correlation  be¬ 
tween  factors 

1 

.05 

The  primary  attributes  that  are  positively  associated  with  the  first 
factor  (Z|)  are  increases  in  speed,  wing  loading,  and,  to  a  lesser  ex¬ 
tent,  the  delivery  efficiency  and  climb  rate.  The  slack  variable  is 
also  described  essentially  entirely  in  terms  of  the  first  factor. 

That  is,  phenomena  occurring  outside  the  development  program  are  cor¬ 
related  positively  with  this  "underlying  dimension."  Overall,  for  the 
sake  of  having  a  label,  I  might  call  this  first  factor  "speed  (with 
ease)."  The  main  idea  behind  this  name  is  that  both  the  speed  and 
wing-loading  dimension  are  quite  strongly  captured  by  the  first  factor. 
More  important  to  the  "(with  ease)"  part  of  the  epithet,  this  factor 
implies  greater  growth  ip  ‘".e  speed  dimension  if  the  slack  period  be¬ 
tween  programs  is  large,  sc  that  advances  not  undertaken  (and  not 
paid  for)  in  the  current  development  program  could  be  embodied  in 
the  current  aircraft.  "Speed  (with  ease)"  doesn't  adequately  capture 
the  extent  to  which  the  first  factor  described  the  climb  rate;  such 
a  name  is  simply  an  abbreviation  for  the  set  of  technical  performance 
improvements  that  are  associated  with  the  factor,  as  detailed  in 
Table  2. 
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C1 early,  both  the  ceiling  and  the  climb  rate  advances  are  very 
strongly  associated  with  the  second  factor.  For  this  factor,  also, 
the  positively  valued  performance  attributes  are  attained  in  part  by 
a  tradeoff  between  weight  and  these  dimensions.  The  speed  and  slack 
variables  are  essentially  not  influenced  by  the  second  factor.  As  a 
label  for  the  second  factor,  I  would  use  a  "ceiling  and  climb1'  char¬ 
acterization.  These  two  factors  are  essentially  uncorrelated,  although 
it  would  be  quite  plausible  that  the  two  factors  described  above  be 
positively  correlated,  since  most  influences  captured  in  a  positive 
manner  by  either  factor  are  considered  desirable  for  advancement  in 
fighter  aircraft.  This  lack  of  correlation  suggests  that  program  ob¬ 
jectives  in  the  first  dimension  can  be  sought  independently  of  objec¬ 
tives  in  the  second. 

By  obtaining  factor  measurements,  I  am  in  a  position  to  test  the 
meaningfi'1ness  of  the  subjective  technological  advance  measure,  A,  in 
the  cost  factor  model.  Specifically,  A  can  be  handled  as  a  variable 
measured  with  noise,  and  the  Z-'s  can  be  used  as  instrumental  variables 
in  an  errors-in-variables  formulation  to  improve  the  estimation  of 
parameters  of  the  cost  factor  model. 

Since  there  were  only  six  fighters  in  the  previous  analysis  (all 

from  the  1950s  sample)  for  which  Z-'s  have  been  obtained,  these  results 

’  * 

are  more  suggestive  than  conclusive.  In  Table  3  the  estimates  of 
the  cost  factor  model  are  presented  for  three  cases:  with  the  original 
subjective  measure  of  A,  with  an  estimated  A  from  a  linear  regression 
on  the  Z^'s  [represented  as  A(Z)3,  and  with  the  latter  measure  of  tech¬ 
nological  advance  and  an  improved  measure  of  program  length  for  each 

system  that  was  used  in  the  calculation  of  the  rates  of  improvement 
** 

in  (4).  The  results,  both  for  the  six  fighters  alone  and  for  the 


The  "A"  measure  for  these  six  does  have  a  broad  range--from  8.5 
to  15.2  (the  entire  1950s  sample  ranges  from  7.0  to  16.0). 

irk 

To  distinguish  these  two  measures  of  program  length,  the  one 
used  in  the  previous  cost  factor  analysis  is  called  "LA;"  the  im¬ 
proved  measure  is  "LZ." 
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Table  3 

COST  FACTOR  MODEL  USING  Z.'s  TO 
IMPROVE  MEASUREMENT  OF  A 


Structure 
Linear  in  " 

A" 

Structure  with  Stretch 
"A"  Dimension  . 

log  F  = 

a  +  b  1 

L  •  A 

miii 

—SB 

Equation 

a 

ll 

m 

mm 

Six  fighter  subsample 

LA  •  f (A) 

-.174 

(-.5) 

.0010 

0.8) 

.45 

,109 

(.5) 

.35E-08 

(2.3) 

.57 

LA  *  f(A(Z) ) 

-.167 

(.5) 

.0010 

0.9) 

.46 

.131 

(.6) 

.22E-08 

(2.4) 

.59 

LZ  •  f (A(Z) ) 

-.285 

(-1.0)1 

.64 

.130 

(.6) 

.18E-08 

(2.4) 

.59 

1950s  aircraft  and  missiles  sample 
LA  •  f (A) 

.021 

(.1) 

■ 

.39 

.268 

(2.2) 

.1 5E-08 
(4.2) 

.48 

LA  *  f (A(Z) ) 

.020 

(.1) 

I] 

.39 

.254 

(2.2) 

.15E-08 

(4.5) 

.52 

LZ  •  f(A(Z) ) 

ti  .  V  .  . . 

-.013 

4.-.0.L 

1  .0008 

1  (3.7) 

.42 

.250 

(?•?) 

•14:-08 

.,(4-l)  , 

.53 

AT 

Mote" :  t-stati sties  are  presented  in' parentheses.  The  equations  in  the'  first 
line  of  the  "1950s  aircraft  and  missiles  sample"  are  reproduced  from  Table  1. 


entire  sample  with  "corrected"  A  measures  in  these  six  cases,  show  that 
the  estimates  for  the  parameters  remain  essentially  unchanged  but  the 
significance  of  the  estimate  of  the  "b"  parameter  increases.  The  one 
case  in  which  the  coefficients  are  quite  different  is  that  of  the 
"stretched"  subjective  A  measure  with  the  six-fighter  subsemple.  In 
this  case  the  "b"  coefficient  is  .0000000035,  double  the  value  estimated 
in  other  analyses  of  the  1950s.  However,  when  the  improved  measures 
are  used--more  accurate  measures  of  program  length  and  estimates  of  A 
from  the  Z^'s— the  coefficient  becomes  nearly  the  same  as  those  esti¬ 
mated  for  the  larger  sample;  that  is,  .0000000018.  In  short,  improve¬ 
ment  both  in  measurement  of  technological  advancement  and  program  length 
lead  to  increased  confidence  in  the  measurement  o.  parameters  of  the 
cost  factor  model  and  therefore  in  its  use  as  a  signaling  device  for  the 
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need  to  devise  alternative  strategies. 


* 


Evaluation  of  Strategy  Elements 

To  deal  with  some  of  the  detailed  elements  of  acquisition  strat¬ 
egies,  it  will  be  necessary  to  abandon  the  cost  factor  representation 
of  program  performance.  The  change  I  will  introduce  involves  the 
premise  that  the  functional  relationship  explaining  cost  outcomes 
based  on  schedule  and  technical  considerations  changes  between  the 
early  planning  for  a  new  system  and  the  time  after  initiation  of  the 
development  program. 

Specifically,  with  efficient  planning,  schedule  stretchouts  be¬ 
fore  development  commences  can  be  traded  off  for  lower  total  program 
costs  indefinitely.  I  have  selected  a  rectangular  hyperbola  as  a 
reasonable  specification  for  such  a  relationship;  see  Fig.  2(a). 
Basically,  the  type  of  "friction"  of  schedule  stretchouts  arguments 
used  to  justify  the  U-shaped  cost  curve  do  not  apply  if  a  development 
program  has  not  proceeded  to  the  point  at  which  a  labor  force  might 
be  forced  to  remain  idle  or  management  coordination  of  a  long  program 
becomes  inefficient.  At  this  early  stage,  it  is  even  feasible  to  post¬ 
pone  initiating  the  development;  present  discounted  costs  decline  as 
program  initiation  is  postponed.  Once  a  program  is  initiated  accord¬ 
ing  to  a  particular  strategy  (e.g.,  planned  time  for  development,  in¬ 
centives  for  meeting  certain  milestones,  etc.),  unforeseen  technolog¬ 
ical  difficulties  or  changes  in  the  system  sought  (due,  for  example, 
to  a  changed  perception  of  threats)  can  lead  to  increased  system  cost 
even  if  handled  efficiently.  I  depict  this  implementation  strategy 

I  have  previously  pointed  out  that  the  e^  stretching  of  the  A 
dimension  may  mean  that  small  technological  advances  have  little  ef¬ 
fect  on  the  unpredictability  of  costs  or  that  the  survey  measure  used 
may  not  be  very  sensitive  to  the  lower  levels  of  technological  advance, 
but  can  only  distinguish  these  from  the  very  highest  [Harman  (1971) 

P.  24]. 

These  resu1ts--while  extremely  limited  because  of  the  sample  size 
involved--provide  the  first  evidence  that  the  latter  interpretation 
may  be  correct.  That  is,  with  this  "quantitative"  measure  of  techno¬ 
logical  advance  (and  better  measure  of  proqram  length),  the  linear 
(more  gradual)  influence  in  the  cost  factor  model  is  now  marginally 
"preferred." 


a)  CONCEPT  FORMULATION  TRADE-OFFS 


c 


b)  STRATEGY  IMPLEMENTATION 

c 


Fig.  2  --  Cost-Schedule  Choice  for  a  Given 
Technological  Advance 
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for  the  program  as  a  parabola  within  the  envelope  of  possible  cost- 
schedule  choices;  see  Fig.  2(b).  A  program  that  can  be  changed  rela¬ 
tively  cheaply  in  response  to  modest  alteration  of  objectives  is  one 

★ 

characterized  by  a  relatively  broad  parabola.  Thus,  the  latue  rectum 
of  the  parabola  can  be  used  as  one  measure  of  the  cost  flexibility  of 
particular  strategies. 

The  formulation  for  the  conceptualization  phase  of  a  new  program, 
then,  is 


(c  -  C.  )  (L  -  L.)  =  k  (5) 

min  '  min  ' 

in  which  C  is,  as  before,  the  total  of  development  and  production  costs 

'k'k 

of  the  new  system,  and  L  is  development  program  length.  Since  the 
model  is  designed  mainly  to  deal  with  outcomes  of  R&D,  it  would  be 
desirable  to  consider  separately  the  technology-advancing  and  cost- 
reducing  aspects  of  the  program.  Unfortunately,  such  a  refined  break¬ 
down  is  simply  not  possible  with  the  available  data.  Therefore,  Cm^n 
is  used  to  allow  crudely  for  the  production  costs  if  no  R&D  were  nec¬ 
essary.  The  height  above  this  level  is  dependent  on  the  ambitiousness 
of  the  R&D  phase;  it  can  be  approximated  by  a  measure  of  technological 
advance  sought,  which  could  influence  both  L  .  and  k.  To  simplify 
the  formulation,  I  assume  that  all  the  influence  of  technological  ad¬ 
vance  can  be  captured  within  k;  that  is,  that  if  R&D  funds  approach 
an  infinite  amount,  the  R&D  time  can  approach  zero.  Restating  equa¬ 
tion  (5),  the  model  during  concept  formulation  becomes 


(6) 


I  should  really  stop  here  for  a  number  of  reasons.  First,  the 
information  I  have  on  the  programs  in  the  1960s  sample  is  extremely 
limited.  Even  if  l  could  estimate  this  hyperbola  (a  question  I  will 
take  up  momentarily),  I  have  only  extremely  crude  information  on  the 


The  latue  rectum  is  the  distance  across  a  parabola  at  the  height 

of  the  focus.  This  is  indicated  by  the  dashed  line  in  Fig.  2(b). 

★  ★ 

This  rectangular  hyperbola  can  be  translated  into  the  first  quad¬ 
rant  by  allowing  the  horizontal  asymptote  (i.e.,  C  .  )  and  the  vertical 
asymptote  (i.e.,  L  -n)  to  be  positive.  in 


K5-C 
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elements  of  strategy  used  for  each  program.  For  example,  Information 
on  the  total  number  of  contractors  and  contracts  is  available,  but  the 
timing  of  the  Individual  decisions  to  initiate  one  or  several  contracts— 
needed  for  an  assessment  of  the  contractor  environment  (e.a.,  sole 
source  or  not)  and  buyer  information  (e.g.,  was  test  information  avail¬ 
able?)— is  not  yet  assembled.  Similarly,  contract  types  are  known 
but  only  at  the  crude  level  of  the  designation:  *ixed-price-incentive, 
cost-plus-incentive-fee,  firm-fixed-price,  and  cost-plus-fixed-fee. 

No  sharing  rates  or  details  of  the  features  upon  which  the  incentives 
are  based  has  been  assembled,  nor  do  I  know  if  contract  "ceilings" 
have  been  exceeded  (at  which  point  the  contract,  however  designated, 
becomes  either  a  firm-price  or  a  cost-plus  type).  I  have  some  information 
on  the  timing  of  the  test  phase,  but  not  on  the  planned  timing  to  be 
related  to  planned  decision  points  for  the  program-most  relevant  to 
the  question  of  "prototype"  versus  "concurrent"  development  strategies 
[see  also,  Klein,  et  al .  (1971)]. 

The  other  major  stumbling  block  to  further  analysis  is  the  prob¬ 
lem  of  estimating  the  hyperbola  (6).  The  case  that  this  cost  equation, 
based  on  planning  considerations  within  the  DoD,  is  part  of  a  block 
recursive  system  would  appear  to  be  rather  weak.  Of  course,  that  only 
involves  the  question  of  the  best  method  of  estimation,  provided  that 
the  equation  remains  identified.  But  the  problem  of  determining  the 
envelope  within  which  all  feasible  program  outcomes  must  occur  demands 
much  more  complete  program  planning  information  and  assessment  of  tech¬ 
nical  difficulty  than  is  now  available. 

Having  said  this,  I  will  briefly  report  some  first  indications  of 
results,  as  a  solicitation  of  your  comments  and  suggestions.  I  have 
used  ordinary  least  squares  to  estimate  equation  (6),  with  the  term 
Q  •  Wt  (quantity,  weighted  by  the  empty  weight  of  the  item  involved  as 
a  proxy  for  production  resources  involved)  to  capture  Cmin,  and  with 
various  transformations  of  the  subjective  A  measure  of  technological 
advance  sought,  for  the  1960s  aircraft  and  missiles  sample.  The  equa¬ 
tion  with  f(A)  =  A2  produced  the  best  fit  (although  no  transformation 
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of  A  yielded  radically  different  explanatory  power): 

C  »  -.64  +  .78  X  10"6  Q  •  wt  +  .71  x  10"6  Q  •  Wtm  +  .79  A2/L  (7) 
(-.3)  (11.5)  ac  (5.9)  m  (1.2) 

R2  *  .94 

A  strategy  parabola  was  then  calculated  for  each  system  based  on  three 
pieces  of  information:  the  planned  cost  and  program  length  [used  in 
estimating  equation  (6)],  the  slope  of  the  hyperbola  [equation  (7)] 
to  which  the  parabola  is  to  be  tangent  at  that  program  length,  and 
the  actual  cost  and  program  length  of  the  completed  system. 

Let  me  conclude  by  indicating  the  kind  of  implications  that  may 
be  feasible  with  this  formulation  of  strategy  influences.  I  have 
plotted  the  lams  recta  of  these  parabolas  against  one  of  the  strat¬ 
egy  elements --the  proportion  of  all  major  contracts  for  a  program 
that  involved  government  cost-sharing  in  the  development  phase  (see 
Fig.  3),  This  choice  of  contract  types  distinguishes  two  things. 
Within  the  development  phase  it  distinguishes  contracts  for  which 
government  agreements  on  the  "target"  and  "allowable"  costs  are  im¬ 
portant.  Li  nee  target  cost  is  less  readily  determinable  for  develop¬ 
ment  activities,  one  would  expect  the  incentive  to  achieve  costs  be¬ 
low  target  to  be  most  easily  perverted  by  the  incentive  to  raise  the 
target  for  these  contracts  [see  also,  Fisher  (1968)].  The  second 
aspect  of  this  proportion  is  that  it  crudely  captures  the  decisions 
and  relative  need  for  development  activities  as  distinguished  from 
production  activities.  That  is,  a  proportion  of  unity  implies  that 
a  comnitment  was  made  for  the  entire  program  at  the  beginning  of  de¬ 
velopment  ("Total  Package  Procurement"),  whereas  a  low  proportion  In¬ 
dicates  either  a  reliance  on  fiim-fixed  price  development  contracts 
or  relatively  little  need  for  development  activities. 

Since  some  programs  had  very  little  change  between  planned  and 
actual  development  program  length  (and  since  the  latue  rectum  will 

- - - 

In  the  equation,  C  is  measured  as  10  $  and  separate  coefficients 
have  been  obtained  for  for  aircraft  and  for  missiles  within  the 

sample.  The  t-statistics  are  presented  in  parentheses  below  the  coef¬ 
ficient  estimates. 


PROPORTION  OF  COST-SHARED  DEVELOPMENT  CONTRACTS 
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approach  zero  as  the  difference  between  planned  and  actual  program 
lengths  goes  to  zero),  the  plot  points  for  programs  with  schedules 
deviating  from  planned  by  more  than  4  months  are  circled.  For  these, 
there  is  no  correlation  between  differences  of  planned  from  actual 
program  length  and  size  of  the  lotus  rectum ,  and  there  does  appear 
to  be  a  pronounced  tendency  for  more  cost-shared  features  to  constrain 
the  cost  flexibility  of  (i.e.,  reduce  the  latus  rectum  of  the  strategy 
parabola  representing)  the  program. 

In  summary,  the  objective  of  this  exercise  has  been  to  suggest 
a  model  formulation  that  may  be  useful  in  identifying  low-cost  acqui¬ 
sition  strategies  based  on  strategy  elements  that  have  been  used  in 
recent  experience.  Of  course,  more  detailed  data  and  many  other  ele¬ 
ments  of  the  strategies  must  be  considered  simultaneously  before  any 
such  inferences  can  be  drawn  with  confidence. 
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